could we use format for this? e.g. format: C#.

pro:

• fewer features to memorize
• less duplication
• less code

con:

• maybe slightly less precise

The problem with overloading the file format feature is that file to language is a one-to-many mapping, e.g. there can be embedded templates that contain multiple different script languages such as C# for server-side scripts and JavaScript for client-side.

we should also think about maybe trying to guess the language based on the file contents if there's no extension.

also supporting things like .cs_ which some may use to prevent the file from accidentally getting executed.

Auto-identifying the script language is definitely a worthwhile feature which I have deprioritized for the minimal implementation and instead manually incorporated the extensions (now including the defanged extensions as suggested above). This is also because it might not always be very straightforward to do so (e.g. one file might include multiple scripts; sometimes context might be necessary).

Can we document why we are only/specially handling ASPX here?

This is related to the issue discussed here: #1080 (comment).

Can we add a comment on what are assumptions are here? I'm not overly familiar with tree-sitter but it appears that we assume it will only throw errors when encountering a language mismatch e.g. we attempt to parse Python using tree-sitter C# tooling?

This might be more readable, what do you think?

def _parse(ts_language: Language, buf: bytes) -> Optional[Tree]:
    try:
        parser = Parser()
        parser.set_language(ts_language)
        return parser.parse(buf)
    except ValueError:
        return None


def _contains_errors(ts_language, node: Node) -> bool:
    return ts_language.query("(ERROR) @error").captures(node)


def get_language_ts(buf: bytes) -> str:
    for language, ts_language in TS_LANGUAGES.items():
        tree = _parse(ts_language, buf)
        if tree and not _contains_errors(ts_language, tree.root_node):
            return language
    raise ValueError("failed to parse the language")

does this mean we only support Linux?

Tree-sitter needs to compile its (C) language bindings. Although I have a limited knowledge of package management, I've suggested to Moritz that we should precompile and package the supported tree-sitter bindings for each platform we support. The current state is a temporary measure.

can this call generate any exceptions?

It can throw type errors (which I believe we prevent with mypy) and a value error if parsing completely fails. Then the parse method will throw a ValueError, so the engine will throw a ValueError etc.: I can handle it in the following way at the Extractor level:

try:
    self.language = capa.features.extractors.ts.autodetect.get_language(path)
    self.template_engine = self.get_template_engine(buf)
    self.engines = self.get_engines(buf)
except ValueError as e:
    raise UnsupportedFormatError(e)

The intended use of this function appears to be decoding a string found in a specific byte range? If so, consider changing the name to something more descriptive like get_str_from_range. Also, do we expect encoding exceptions to be thrown by the decode?

Good question, but I doubt tree-sitter would be able to parse something that we can't decode.

Also changed get_range to get_str.

need more discussion on #1125

typo w/ _?

No, this is merely a private method to handle import table lookups that the public is_import method uses.

do we expect exceptions to occur here?

This can raise ValueError (and does when TS labels something as an int which is not an int), but the ValueError is handled by the caller (see extract_integers).